Composition and Arrangement Techniques for Music in Interactive Immersive Environments

Inspired by the dramatic and emotional effects of film music, we aim at integrating music seamlessly into interactive immersive applications — especially in computer games. In both scenarios it is crucial to synchronize their visual and auditory contents. Hence, the final cut of movies is often adjusted to the score or vice versa. In interactive applications, however, the music engine has to adjust the score automatically according to the player’s interactions. Moreover, the musical effects should be very subtle, i. e., any asynchronous hard cuts have to be avoided and multi-repetitions should be concealed. This paper presents strategies to tackle the challenging problem to synchronize and adapt the game music with nonpredictable player interaction behaviors. In order to incorporate expressive scores from human composers we extend traditional composition and arrangement techniques and introduce new methods to arrange and edit music in the context of interactive applications. Composers can segment a score into rhythmic, melodic, or harmonic variations of basic themes, as known from musical dice games. The individual parts of these basic elements are assigned to characterize elements of the game play. Moreover, composers or game designers can specify how player interactions trigger changes between musical elements. To evaluate the musical coherency, consistency, and to gain experience with compositional limitations, advantages and possibilities, we applied this technique within two interactive immersive applications.